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Claims 

1 . A simultaneous music recording apparatus for a video recorder that is a simultaneous 
music recording apparatus that controls the recording of music that plays simultaneously with the 
recorded image of the recording apparatus, and which is equipped with a performance means that 
automatically plays songs based on song performance data, a control to instruct the performance 
means to start performance, a control signal transmission means that generates a control signal that 
expresses the instruction to start music performance corresponding to the operation of the control, 



* [Numbers in the margin indicate pagination of the original foreign text.] 



and transmits said control signal to the video recorder so that the video recorder records said 
control signal at the recording position corresponding to the image being recorded at the point at 
which the start performance instruction was given, a first control means that performs control 
during video recording to instruct the performance means to start performance corresponding to 
operation of the control, and so that performance by the performance means is stopped in 
synchronization with the video recorder stopping recording, and so that performance by the 
performance means restarts from the stop position in synchronization with the video recorder 
starting recording, and a second control means that performs control during video playback so that 
performance by the performance means is started according to the control signal that is read out 
together with the video signal from the video recorder. 

2. A simultaneous music recording apparatus for video recorder that is a simultaneous 
music recording apparatus that controls the recording of music that plays together with the 
recorded image of the recording apparatus, and which is equipped with a performance means that 
automatically plays songs based on song performance data, a control to instruct the performance 
means to start performance, a performance data transmission means that sequentially transmits 
performance data for a song being played by the performance means to the video recorder so that 
the video recorder records the song performance data at the recording position corresponding to 
the video image being recorded during performance by the performance means, and a control 
means that performs control to stop performance by the performance means in synchronization 
with the video recorder stopping recording, and restart performance by the performance means 
from the stop position in synchronization with the video recorder starting to record. 

Detailed explanation of the invention 
[0001] 

Technical field of the invention 

The present invention pertains to a simultaneous music recording apparatus for video 
recorders that is linked with a video recorder, such as the video recorder built into a video camera, 
and which records background music, etc. to be played with the recorded video image 
simultaneously with the video recording. 

[0002] 
Prior art 

Normally, to have background music (hereinafter abbreviated BGM) play together with 
images taken with a video camera with built-in video recorder (hereinafter, video camcorder), the 
BGM is recorded onto an audio track at the same time that the video is being shot, and a common 
method used for this kind of recording is to carry an audio CD player or cassette player together 



with the video camcorder and simultaneously play music on the audio player while shooting with 
the video camcorder, mixing the input audio from the external microphone on the video camcorder 
to record the playback audio on the video camcorder audio track. 

[0003] 

Problems to be solved by the invention 

Shooting with a video camcorder normally progresses by shooting one scene, temporarily 
pausing the video camcorder, and then shooting the next scene after some time has lapsed, and 
ultimately playing back these multiple scenes linked together as a single video scene during 
playback. The background music then plays as continuous music throughout the entire video of the 
various scenes. 

[0004] 

In order to realize this with the past recording method, it was necessary to shoot video by 
simultaneously stopping the audio player whenever the video camcorder was stopped when 
shooting one scene, and then run the audio player simultaneous with the video camcorder when 
shooting the next scene in order to produce continuous playback music. 

[0005] 

However, when this recording method was performed using this technique, it was difficult 
to perfectly match the timing of starting/stopping the video camcorder and audio player due to the 
complexity of the operation, so that the background music was not continuous at the scene 
transitions during playback of the recorded video. 

[0006] 

Structures that would automatically synchronize the starting/stopping of the video 
camcorder and audio player were also considered, but it would be difficult with the existing 
technology to avoid producing noise at the transitions of starting/stopping the audio player, which 
noise would get mixed into those transitions during playback of the recorded video as "pops'* in the 
background music. 

[0007] 

The purpose of this invention is to address these problems so that BGM, etc. can be 
simultaneously recorded during video shooting, and music can continue seamlessly even while 
scenes are changing. 



[0008] 

Means to solve the problems 

Figure 1 is an explanatory drawing of the basic principles of the simultaneous music 
recording apparatus for video recorder of this invention, which controls the recording of music that 
plays together with the video recorded by the video recorder. In order to solve the problems 
described above, one embodiment of the simultaneous music recording apparatus for video 
recorder of this invention is equipped with a performance means that automatically plays songs 
based on song performance data, a control to instruct the performance means to start performance, 
a control signal transmission means that generates a control signal that expresses the instruction to 
start music performance corresponding to the operation of the control, and transmits said control 
signal to the video recorder so that the video recorder records said control signal at the recording 
position corresponding to the image being recorded at the point at which the start performance 
instruction was given, a first control means that performs control during video recording to instruct 
the performance means to start performance corresponding to operation of the control, and so that 
performance by the performance means is stopped in synchronization with the video recorder 
stopping recording, and so that performance by the performance means restarts from the stop 
position in synchronization with the video recorder starting recording, and a second control means 
that performs control during video playback so that performance by the performance means is 
started according to the control signal that is read out together with the video signal from the video 
recorder. 

[0009] 

This simultaneous music recording apparatus operates linked with the video recorder, 
whereby the operator uses the control to start automatic performance by the performance means 
at the indicated song performance timing and send a control signal indicating the start of 
performance to the video recorder. The video recorder records the control signal at the recording 
position corresponding to the video image in the recording at the time at which the start of 
performance was indicated. Furthermore, during video recording, the first control means stops 
performance by the performance means in synchronization with the video recorder stopping video 
recording, and restarts performance by the performance means at the stop position in 
synchronization with the video recorder starting video recording. The second control means starts 
performance by the performance means according to the control signal that has been read out 
together with the video recording signal by the video recorder. This allows the song that was 
playing as the background music during video recording to be automatically played back during 
playback of the video recording with the same correlation between the recorded video image and 



the background music, moreover allowing it to be heard as continuous music during playback, 
regardless of the video recorder being started and stopped during video recording. 

[0010] . 

The simultaneous music recording apparatus for video recorder of this invention, as 
another embodiment, is equipped with a performance means that automatically plays songs based 
on song performance data, a control to instruct the performance means to start performance, a 
performance data transmission means that sequentially transmits performance data for a song 
being played by the performance means to the video recorder so that the video recorder records the 
song performance data at the recording position corresponding to the video image being recorded 
during performance by the performance means, and a control means that performs control to stop 
performance by the performance means in synchronization with the video recorder stopping 
recording, and restart performance by the performance means from the stop position in 
synchronization with the video recorder starting to record. 

[0011] 

The performance means automatically plays back a song based on the performance data of 
the song, and the performance data transmission means sequentially transmits performance data 
for the song being played. The video recorder records the performance data for the song being 
played by the performance means at the recording position that corresponds with the video image 
being recorded as it is played. The control means performs control to stop performance by the 
performance means in synchronization with the video recorder stopping video recording, and 
restarts the performance by the performance means from the stop position in synchronization with 
the video recorder starting video recording. In this way, the performance data for the song that was 
playing as background music during video recording is recorded in the recorded video image on 
the video recorder in correlation with the video image at each point in time. Thus, if these 
performance data are read together with the video signal and automatically played by the 
performance means during playback, the song that was playing as background music during video 
recording can be automatically played during video playback with the same correlation between 
the recorded video image and the background music as during video recording, and moreover can 
be heard as continuous music during playback, regardless of video recording by the video recorder 
being started and stopped during video recording. 



[0012] 

Embodiments of the invention 

Embodiments of this invention will be described below, referring to the attached figures. 
Figure 2 shows the block structure of a simultaneous music recording apparatus for a video 
recorder (hereinafter referred to in the embodiments of this invention simply as a simultaneous 
recorder) as an embodiment of this invention. Figure 3 also shows an example of the condition of 
mounting the simultaneous recorder to the video camcorder. As shown in Figure 3, the 
simultaneous recorder 3 1 is attached and mounted by means of a screw 33 on the bottom of the 
video camcorder 30. In addition, signal lines are connected between input terminals and output 
terminals on each. An earphone may also be connected to this simultaneous recorder 31, as 
needed. 

[0013] 

As shown in Figure 2, the simultaneous recorder 31 is equipped with input and output 
terminals with output terminals 13,14 for the left and right audio signals, input terminals 15, 16 for 
left and right input signals, a video signal input terminal 17, an earphone output terminal 18, and a 
mixed audio output terminal 19, and signal lines are connected so that left and right audio signals 
output from the output terminals 13, 14 are input to the left and right audio input terminals on the 
video camcorder 30, and the left and right audio signal and the video signal from the video 
camcorder 30 are respectively input to the audio input terminals 15, 16 and video input terminal 
17. 

[0014] 

The various structural elements of this simultaneous recorder 3 1 will be described below. 
The controller 1 is a circuit constructed to include a CPU (central processing unit) that enables 
control of the entire device, and controls the operation of a sequencer 3 according to the operation 
of a control group 2 and starting and stopping of the video camcorder 30, as will be described 
below. 

[0015] 

The control group 2 comprises selector buttons that select and play the BGM that is to play 
with the recorded video image, a stop button that stops performance, and a mode selection switch. 
Performance of a BGM song is started at the same time that one of these selector buttons is pressed 
to select a BGM song. 



[0016] 

The sequencer 3 is controlled by the controller 3 to start performance of a specified BGM 
song (START), stop performance (STOP), and continue-start performance (CONTINUE-ST ART) . 
This sequencer 3 is constituted to notify the controller 1 of its status whether or not it is playing. 
The sequencer is also constituted to store the position at which performance is stopped when a stop 
performance instruction is given, and to restart performance from that stored stop position when an 
instruction is given to continue-start performance. "Continue-start" (CONTINUE-START) in this 
embodiment refers to the operation of restarting performance from the position at which 
performance was stopped. 

[0017] 

A song data ROM 4 stores a multiplicity of data for BGM songs to be played by the 
sequencer 3 (performance data). These performance data are recorded as MIDI data in this 
embodiment. A sound source 5 actually generates music by driving the sequencer 3. In this 
embodiment, music is generated according to a MIDI signal transmitted from the sequencer 3. 

[0018] 

A microphone 10 is for recording external sound, and recorded sound is supplied to the 
audio input terminal (right) on the video camcorder 30, and supplied to a mixer 6, via an amplifier 
1 1 . The mixer 6 is a circuit that mixes and then externally outputs the sound captured by the 
microphone 10 or the audio output (right) from the video camcorder 30 with the BGM from the 
sound source 5. 

[0019] 

The earphone output terminal 18 is a terminal for connecting an earphone for monitoring 
the mixed sound of the BGM sound from the sound generator 5 and the input sound from the 
microphone 10 that is output by the mixer 6. In addition, the audio output terminal 19 is an 
output terminal to output the mixed sound of the BGM sound and original audio (playback audio 
from the video camcorder) output from the mixer 6 during playback. 

[0020] 

An encoder 8 is a circuit that converts the control signal for simultaneous recording of 
BGM (hereinafter abbreviate as the simultaneous recording signal) that is transmitted from the 
controller 1 to a signal in a format that can be recorded on the audio track CHI of the video 
camcorder 30 (e.g., FSK (frequency shift keying) signal). In addition, a decoder 9 is a circuit that 
decodes the simultaneous recording signal recorded on the audio track CHI of the video 



camcorder 30 that had been converted by the encoder 8 to its original format and supplies it to the 
controller 1 . Thus, in this embodiment, of the two audio tracks CHI, CH2 on the video camcorder 
30, the left channel audio track CHI is used as the control track for simultaneous recording of the 
BGM. 

[0021] 

A synchronization detection circuit 7 is a circuit that detects the presence of 
synchronization signal (vertical/horizontal video synchronization signal) from the video output 
signal of the video camcorder 30 and notifies the controller 1. The controller 1 is able to determine 
on the basis of this notification that the video camcorder 30 is operating if there is a 
synchronization signal, or that it is stopped if there is none. 

[0022] 

A detail of the control group 2 is shown in Figure 4. As shown in this figure, the switch 
buttons numbered (1-8) are selector buttons, and a variety of BGM songs corresponding to these 
numbers 1 through 8 are stored in the song data ROM 4 described below. Pressing these selector 
buttons during video recording selects a BGM song and sends an instruction to start BGM 
performance. The button labeled [STOP] is the stop button, which is pressed to stop the BGM 
being played. The switch labeled [MODE] is a switch that switches the operating mode of the 
simultaneous recorder, and can be set to "REC" during video recording or "PLAY" during 
playback. 

[0023] 

Next, an overview of the operation of the simultaneous recorder of this embodiment will be 
described, referring to Figure 5. Figure 5 is an explanatory diagram that shows the condition of 
recording the start signal and active signal on the audio track CHI in the recording track of the 
video camcorder 30 (comprising the video track and left and right audio tracks) together with the 
operation of the sequencer 3 on the time axis, wherein (A) shows a circumstance during video 
recording and (B) shows a circumstance during playback. Namely, (A) shows a circumstance in 
which recording was paused during video recording after scene A was recorded on the video track, 
and then recording of scene B was restarted after some time had lapsed, and (B) shows a 
circumstance in which scene A and scene B recorded on the video track are continuously played 
back. 



[0024] 

(Video recording operation) First, the video recording operation will be described. In this 
case, the mode selection switch on the simultaneous recorder 31 is set to "REC". During video 
recording with the video camcorder 30, the scene captured by the camera is recorded on the video 
track and the sound input from the microphone 10 is recorded on the audio track CH2. 

[0025] 

(1) When video recording by the video camcorder 30 is started, the video signal output 
from the video camcorder 30 is input to the simultaneous recorder 3 1 and provided to the 
synchronization detection circuit 7. When the synchronization detection circuit 7 detects a 
synchronization signal, it notifies the controller 1 that "there is a synchronization signal," and the 
controller 1 determines that the video camcorder 30 is recording video. 

[0026] 

(2) When one of the selection buttons in the control group 2 is pressed during video 
recording by the video camcorder 30, the controller 1 indicates the song number selected by the 
selection button to the sequencer 3 and instructs it to start performance. The controller 1 also sends 
a start signal containing the song number information to the encoder 8. 

[0027] 

(3) The encoder 8 converts the start signal transmitted from the controller 1 into a signal 
(FSK signal) that can be recorded on the audio track of the video camcorder 30 and transmits it to 
the input terminal for audio track CHI on the video camcorder 30. 

[0028] 

(4) The video camcorder 30 records the signal from the encoder 8 to the audio track CHI 
(left) in real time. Thus, at the point at which the selection button was pressed while recording 
video scene A, a start signal is recorded at the position on audio track CHI corresponding to the 
video image at that point. In this way, a start signal and an active signal, explained below, are 
recorded on the audio track CHI as simultaneous recording signals (signals used to control 
operation of the sequencer during video playback). 

[0029] 

(5) Meanwhile, external audio from the microphone 10 is recorded on audio track CH2 

(right). 



[0030] 

(6) In addition, the performance sound of the BGM song from the sound source 5 and the 
external audio from the microphone 1 0 are mixed by the mixer in the simultaneous recorder 3 1 and 
output as audio. The user can monitor this in the earphone 32. 

[0031] 

(7) When a selector button is pressed and performance of the BGM song is started by the 
sequencer, the controller 1 periodically transmits an active signal to the encoder 8. This active 
signal is also sent through the encoder 8 to the video camcorder 30 and is recorded there on an 
audio track CHI. Thereafter, active signals are recorded at a set time interval. This active signal is 
a signal that indicates that a BGM song is being played during recording, i.e. that a BGM song will 
be played during playback of the video image. 

[0032] 

(8) When video recording by the video camcorder 30 is paused, a synchronization signal is 
no longer detected by the synchronization detection circuit 7, and the controller 1 is notified that 
"there is no synchronization." When the controller 1 determines that video recording by the video 
camcorder 30 has been interrupted, it instructs the sequencer to stop playing. 

[0033] 

(9) When video recording by the video camcorder 30 is restarted to record scene B, the 
synchronization detection circuit 7 again detects the synchronization signal, whereby the 
controller 1 determines that the video camcorder 30 is recording and instructs the sequencer 3 to 
resume playing. The sequencer restarts playing, continuing from the position at which it stopped 
playing when recording of scene A was paused. In other words, the controller performs control to 
match the operation of the video camcorder 30 and operation of the sequencer 1 . 

[0034] 

(10) Meanwhile, if the stop button in the control group 2 is pressed during video recording, 
as shown in scene B of 5(A), the controller 1 stops the sequencer 3 playing and stops transmitting 
the active signal that it had been periodically sending up until then. Thereafter, the active signal is 
not recorded on the video camcorder 30 audio track CHI . In other words, audio track CHI is blank' 
from that point on until the next time a selector button is pressed. 



[0035] 

The segment on which an active signal is thus continuously recorded is the segment on 
which BGM was played during video recording, and the point at which this active signal was cut is 
the point at which the stop button was pressed during recording and performance of the BGM was 
stopped. The presence of this signal is monitored during playback and driving of the sequencer 3 is 
controlled so that the BGM song is played/stopped in the same condition as during video 
recording. 

[0036] 

(Operation during playback) The operation during playback will be described next. In this 
case, the mode selection switch on the simultaneous recorder is set to "PLAY". 

[0037] 

(1) When video images are played back on video camcorder 30 for the first time, the video 
track image signal and audio signals on the left and right audio tracks CHI, CH2 are reproduced, 
of which, the audio signal on audio track CH2 (right) is input to the mixer 6 and the simultaneous 
recording signal on audio track CHI (left) is input to the decoder 9. The decoder reconverts the 
simultaneous recording signal that was recorded on audio track CHI and sends the resulting start 
signal to the controller 1 . 

[0038] 

(2) The controller extracts the song number from the start signal that has been sent to it and 
transmits that to the sequencer 3, and instructs the sequencer 3 to start performance of the BGM 
song of that song number. Thereafter, the sequencer 3 will continue to play as long as the active 
signal is not cut. 

[0039] 

(3) Meanwhile, the audio on audio track CH2 is mixed with the BGM audio from the sound 
source 5 and output as audio. In other words, the sound recorded by the microphone 10 during 
recording and the BGM performance sound are heard together as mixed audio. 

[0040] 

(4) After the start signal, the active signal is periodically provided from audio track CHI, 
through the decoder 9, and to the cohtroller 1 . During the time that this active signal is being sent 
without interruption, the controller 1 causes the sequencer 3 to continue to play the BGM. 



[0041] 

(5) When playback of audio track CHI reaches the location at which the stop button was 
pressed during audio recording and transmission of the active signal was stopped (the place at 
which playing of the BGM was stopped), the active signal stops being sent to the controller 1. The 
controller 1 waits a specified time from when the last active signal was sent, and when it detects 
that the next active signal has not come, the controller 1 instructs the sequencer 3 to stop playing. 

[0042] 

(6) If the controller waits for the next start signal to come and receives a start signal, 
operation is repeated from (1). 

[0043] 

Since the time between scene A, when recording was paused during video recording, and 
scene B is zero during video playback, the point at which video recording was paused and the point 
at which it was restarted are connected, and therefore scene A and scene B are continuously played 
back. In precisely the same way, the point at which the sequencer 3 was stopped playing and the 
point at which performance was restarted match at the transition between scene A and scene B 
during audio recording, the segment of audio track CHI corresponding to scene A and the segment 
corresponding to scene B are tied together during playback so that the active signal is periodically 
reproduced, whereby the sequencer plays so that performance is continuous even at the transition 
between scene A and scene B, maintaining the synchronization between the progress of sequencer 
performance and the progress of the video images of the scenes. 

[0044] 

Next, details of the processing by the controller will be described referring to the flow 
charts in Figures 6-12. These flow charts detail the processing in the CPU in the controller 1. The 
various figures will be described below. 

[0045] 

Figure 6 illustrates a routine that is executed first when the power is turned on. The 
variables used for the subsequent processing are initialized and the sequencer is initialized, etc. as 
initialization processing (step A). • 

[0046] 

Next, a timer interrupt routine is started to periodically count up the time counter t. The 
interrupt period of the internal timer (not shown) in the controller 1 is specified and the disable 



interrupt flag is canceled. Thereafter, the timer interrupt routine shown in Figure 7 is periodically 
executed at the designated interrupt period (step B). For instance, if 10 ms is specified as the 
interrupt period, the routine in Figure 7 is executed every 10 ms. 

[0047] 

The status of the mode selection switch is then checked (step C), and if the mode selection 
switch is on the "REC" side, processing moves to a REC routine (step D), or moves to a PLAY 
routine if it is on the "PLAY" side (step E). 

[0048] 

The timer interrupt routine in Figure 7 described above is executed every time a timer 
interrupt is generated. First, the value of variable t is incremented by 1 (step B 1 ). Then, variable t is 
compared to a specified threshold value W so that it does not overflow (step B2), and if variable t 
exceeds the threshold value W, the threshold value W is substituted for variable t (step B3). 

[0049] 

Figures 8 through 10 are the REC mode routine that is executed in the "REC" mode. The 
variables used in this routine are the play flag, continue-start flag, and interval time. The 
significance of these variables is explained below. 

[0050] 

The play flag is a flag that expresses whether the sequencer is playing or not playing, and is 
set to "1" if the sequencer is instructed to start or continue-start playing, or to "0" if it is instructed 
to stop playing. 

[0051] 

The continue-start flag is a flag that indicates whether to enable continue-start, and is set to 
"1" when it is enabled and "0" when it is disabled. In other words, when stop playing is indicated 
due to video recording being stopped (synchronization signal is cut), it is set to "1" to enable 
continue-start. When stop playing is indicated due to the stop button being pressed, it is left at "0" 
so that continue-start is disabled. 

[0052] 

Interval time T int stores the time interval at which the active signal is sent. In this case, 
when the interrupt routine in Figure 7 is called at an interval o f 10ms, if T int =10, an active signal 
will be sent every 100ms. This value is a constant. 



[0053] 

The details of the REC mode routine will be explained below, following Figures 8-10. 
When the REC mode routine is started, first the play flag and continue flag are each initialized to 
"0" (step Dl). Next, there is a check for notifications from the synchronization detection circuit 7 
(step D2) to determine whether there is a synchronization signal (i.e., if video recording is on or 
paused) (step D), and then the status of the play flag is determined according to the presence of a 
synchronization signal ("yes" or "no") (steps D4, D23). 

[0054] 

The flow of processing significantly varies according to the presence of this 
synchronization signal and the status of the play flag, branching into the following 4 cases. 

[0055] 

(Case 1) Synchronization signal "yes", play flag = "1" Branch to step D5 

Video recording is underway and the sequencer is playing. 

(Case 2) Synchronization signal "yes", play flag = "0" Branch to step D13 

Video recording is underway and sequencer performance is stopped. 

(Case 3) Synchronization signal "no", play flag = "1" Branch to step D24 

The sequencer is playing and video recording has been paused. 

(Case 4) Synchronization signal "no", play flag = "0" Branch to step D28 

Sequencer performance is stopped and video recording has been paused. 

[0056] 

The flow chart will be explained below in the sequence of video camcorder 30 operation. 
First, assume that video recording by the video camcorder 30 is started without a BGM song 
playing. In this case, since the synchronization signal is "yes" and the play flag = "0", processing 
branches to case 2 (sequencer performance is stopped and video recording is underway). 

[0057] 

(Case 2) First, there is a check for whether a selector button in the control group 2 has been 
pressed (step D13), and if one has been pressed, the following sequence of processing is executed 
Namely, the song number corresponding to the number of the selector button that was pressed is 
acquired (step D14), and the a start signal containing that song number as data is sent to the 
encoder 8 (step D15). Meanwhile, the song to be played is specified by sending the song number to 
the sequencer 3 and the sequencer 3 is instructed to start playing (step D16). Thus, the sequencer 3 



starts playing the selected performance song. Next, variable t is initialized to "0" in order to 
periodically generate an active signal from the point at which performance was started (step D17). 
In addition, the play flag is set to "1" to indicate that the sequencer 3 is playing. Thereafter, 
processing returns to detecting the presence of a synchronization signal (step D2) and subsequent 
processing is repeated (hereinafter referred to as loop processing) to monitor changes in status. The 
above processing sets up a status in which the video recording by the video camcorder 30 is 
underway and a BGM song is playing. 

[0058] 

After returning from this (case 2) to step D2 of detecting the presence of a synchronization 
signal, since the play flag is determined to be "1" at step D4, subsequent loop processing branches 
to case 1 (sequencer is playing and video recording is underway). 

[0059] 

(Case 1) First, variable t is compared with the interval time T int to determine if t > T int (step 
D5), and if the answer is "yes", an active signal is set to the encoder 8 (step D6) and variable t is 
reset to "0" (step D7). If it is "no", processing regarding the transmission of this active signal jumps 
to the following processing. Since variable t is periodically incremented by one by the timer 
interrupt routine, this processing is repeated as a loop, whereby a start signal has been sent, and 
then an active signal is periodically sent (every time t > Ti nt ). 

[0060] 

In the subsequent processing, it is checked whether the user has pressed the stop button to 
stop the BMG song (step D8). If that button is pressed, an instruction of stopping the playback is 
sent to the sequencer (step D9), and the play flag is set to "0" indicating stop of the playback (step 
D10). In this way, playback of BMG song is stopped even when video movie 30 is being recorded. 
If the stop button is not pressed, the process goes to the next step without carrying out the 
processing regarding stop of the playback. 

[0061] 

In the subsequent processing, a check is made for notification from the sequencer 3 (step 
Dl 1), and if the sequencer 3 has finished playing the song, the play flag is set to "0" to indicate that 
performance has been stopped. This creates a status in which video recording by the video 
camcorder 30 is underway, but the BGM song has finished playing. If performance has not been 



finished, the processing for case 1 is ended without performing processing related to stopping 
performance (i.e., with the play flag left as it was), processing returns to the step of detecting the 
presence of the synchronization signal, and loop processing is repeated. 

[0062] 

Once the video camcorder 30 has operated as described above and video recording is 
stopped, the synchronization detection circuit 7 is no longer able to detect a synchronization signal. 
Thus, the synchronization signal is determined to be "no" in the step to determine detection of the 
presence of a synchronization signal (step D2). When video recording is stopped while a BGM 
song is playing, since the play flag is set to "1" to indicate that it is playing, processing branches to 
case 3 (sequencer is playing and video recording has been paused) (step D23). 

[0063] 

(Case 3) When the video camcorder 30 stops recording video, the controller 1 instructs the 
sequencer 3 to stop playing (step D24) and stores the value for the time variable t in a register as 
stored value t 0 (step D25). The continue-start flag is then set to "1" to indicate that continue-start is 
enabled (step D26) and the play flag is set to "0" to indicate that performance is stopped (step D27). 
Thus, preparations are made for when performance is restarted the next time a synchronization 
signal comes (video recording is resumed). Loop processing is then repeated, returning to step D2, 
to monitor for a synchronization signal. 

[0064] 

When video recording by the video camcorder 30 is resumed from this aforementioned 
state, after setting synchronization signal to "Yes" and the play flag = "0", processing moves to 
(case 2). 

[0065] 

If it is determined at step Dl 3 in the case of (step 2) that a selector button has not been 
pressed, processing branches to a different flow from that described above to determine the status 
of the continue-start flag (step D18). Because this continue-start flag has been set to "1" by the 
processing in (case 3) to indicate that continue-start has been enabled, the controller 1 instructs the 
sequencer 3 to continue-start (step D19). Thus, the sequencer 3 resumes playing the interrupted 
BGM song in synchronization with the resumption of video recording. In addition, the stored value 
to stored in the register is reset to the variable t (step D20) so that the active signal is periodically 
generated after restarting (i.e., so that it is periodically generated during playback from the time the 
last active signal was generated before play was paused). Next, the continue-start flag set to "0" to 



indicate that continue-start is disabled and processing returns to processing to detect the presence 
of the synchronization signal (step D2). 

[0066] 

On the other hand, in the case the operation to play the BGM song is not performed after 
resuming video recording, the continue-start flag remains set to "0" as it was when it was 
initialized at step Dl. Consequently, when the continue-start flag is determined to be "0" at the 
decision at step D18, above, processing in (case 2) is stopped in that condition and returns to the 
processing to detect the presence of the synchronization signal (step D2). 

[0067] 

Because the play flag is set to "0" when the operation to play a BGM song is not performed 
during video recording, or if a BGM song is stopped playing by pressing the stop button, if video 
recording is topped in this condition, the synchronization signal is set to "No M and the play flag is 
set to "0", and processing branches to case 4. 

[0068] 

(Case 4) In this case, in which neither the sequencer 3 nor the video camcorder 30 is 
operating, the mode selection switch is checked (step D28) and if there is no change in the mode (if 
it stays in the "REC" mode), processing returns to detecting the presence of the synchronization 
signal (step D2) without changing anything. If the mode is switched to "PLAY", processing moves 
to the PLAY routine described below. 

[0069] 

Figures 1 1 and 12 show the PLAY mode routine, which executes the operation of the 
simultaneous recorder during video playback, i.e., when in the PLAY mode. The roles and 
significance of the variables used in this routine are the same as they were in the REC mode routine 
described above. 

[0070] 

This is described below as the process of rewinding the video recorded by the video 
camcorder 30 to the beginning and playing back the video. If the mode selection switch on the 
simultaneous recorder is switched to the "PLAY" mode, this PLAY mode routine is executed, but 
the play flag is initialized to "0" (step El). 



[0071] 

There is a check to see whether a signal is coming from the decoder 9 (step E2) and if there 
is a signal, that signal is checked to see if it is a start signal (step E7). If it is a start signal, the 
following processing starts performance of the BGM song from that position. Namely, the status 
of the play flag is checked (step E8). Since the play flag = "0" at this point, performance of the 
BGM song has not yet started, and therefore, the song number information contained in the start 
signal received from the decoder 9 is extracted (step E9) , that song number is sent to the sequencer 
3 to designate the song to be played, and then the sequencer 3 is instructed to start playing (step 
E10). Thus, the sequencer 3 starts to play the BGM song. Furthermore, the play flag is set to "1" to 
indicate that performance is underway (step Ell). 

[0072] 

Since the play flag = "1" when the play flag status is subsequently checked (step E14), 
detection of the presence of a synchronization signal is performed next (step El 6). Since there is 
no video signal, because video playback has been stopped or because a segment is being played on 
which no video signal was recorded, etc., if there is no synchronization signal, an instruction to 
stop playing is given to the sequencer 3 and the play flag is set to "0". After this processing, or if 
the synchronization signal = "Yes", the sequencer 3 is further checked to see if it has finished 
playing a song (step E20). If it has finished, the play flag is set to '"0". After this processing, or if 
the song is not finished playing, processing returns to monitoring the signal coming from the 
decoder 9 (step E2). 

[0073] 

After a start signal has been received, the signal sent from the video camcorder 30 is an 
active signal. Therefore, when it is determined that a signal has arrived from the decoder 9 (E2), 
and that signal is not a start signal (step E7), the signal is cheeked to see if it is an active signal (step 
E12). If it is an active signal, the variable t, which monitors the period at which the active signal 
arrives, is set to "0" to automatically stop performance (step 13). While an active signal is 
periodically arriving, the flow at step El 2 is repeated so that sequencer 3 plays the BGM song in 
parallel with the video playback. 

[0074] 

If the signal that has arrived is neither a start signal nor an active signal, execution 
thereafter returns to step El 4 without anything being done. Furthermore, since this simultaneous 
recorder handles nothing other than the two start and active signals, and essentially no other 



signals will come to it, the aforementioned processing to check for the active signal (step El 2) is 
very certain. 

[0075] 

Thereafter, an active signal arrives periodically and the BGM song is played, but since this 
means that the BGM song will stop during video playback when the active signal stops, processing 
is performed in this case to stop the sequencer 3 from playing. This processing will be explained 
next, according to the flow chart. 

[0076] 

When no signal comes from the decoder 9 (step E2), the variable t is compared with 
interval Ti nt to determine if t > 2 • Ti nt (step E3). If the decision result is YES, since more than a 
specified time (2 • Ti nt ) has passed since the last active signal arrived, it is determined that the 
active signal was stopped during video recording at that performance position. Therefore, the play 
flag status is checked (step E4), and if it is "1" to indicate that BGM was being played, the 
sequencer 3 is instructed to stop playing (step E5) and the play flag is set to "0" (step E6). 

[0077] 

In other words, in a case where the sequencer is playing and an active signal, which was 
supposed to have been sent at the time interval T^t, has not arrived even after waiting the time (2 * 
Tint) (i e., when there was a 2 • Ti nt -long blank on the video camcorder 30 audio track CHI), the 
controller 1 sees this as a signal to stop the performance and stops the sequencer 3. 

[0078] 

In a case when the sequencer 3 has been stopped, if the BGM song was finished playing, or 
if it was not being played from the start, etc., the play flag is set to "0". In this case, the play flag = 
0 determination is made at step El 4, and the status of the mode selection switch is checked at the 
next step (step El 5). If the mode has not been switched (if it is still in "PLAY" mode), execution 
moves to the next step (step E20) without changing anything. If the mode has been switched to 
"REC", execution moves to the REC mode routine described above. 

[0079] 

When the processing in the segment from step E14 through El 9 is finished, a check is 
made for a notification from the sequencer 3 (step E20) and if the sequencer 3 is finished playing a 
song, the play flag is set to "0". 



[0080] 

A variety of modifications are possible in the embodiment of this invention. For example, 
in the embodiment described above, an active signal is periodically sent while a BGM song is 
being played during video recording, but this invention is not limited to this, and, instead of 
periodically transmitting an active signal, a pause play signal or stop play signal could be 
respectively written to the video camcorder audio track when BGM performance is paused or 
stopped, and the sequencer could be controlled according to these signals during playback. 

[0081] 

In addition, only the selected song number was included and sent in the start signal for that 
song in the embodiment described above, but the song number also could be periodically recorded 
on the audio track in combination with the time lapsed from the start of playing to enable the video 
camcorder to play a video from the middle. Thus, even if the video camcorder were to play a video 
from the middle, the sequencer is able to refer to the readout information and play from the 
location in the BGM song corresponding to that midway starting position. 

[0082] 

An audio track was used in the embodiment described above as the track for recording the 
signals to control simultaneous recording, but this invention is not limited to this and these could 
also be recorded on a control track provided in addition to the video track and audio tracks. 

[0083] 

Additionally, only data indicating to start performance and stop performance were 
recorded in the embodiment described above, but the performance data (MIDI data) themselves 
could be recorded instead. If the BGM were thus recorded as the performance data, the sound 
source could be driven by the values of the performance data read out during playback, enabling 
continuous performance without any noise at the scene cuts. 

[0084] 

The circumstances in which a signal corresponding to the performance data is recorded on 
audio track CHI is shown in Figure 13. In this condition, these signals are used in pairs wherein a 
note generated at "Note ON" is silent at M Note OFF". Figure 13(a) shows an instance in which 
video recording is paused at the point at which Note ON for music "D" is written to the audio track, 
and subsequently Note OFF is written for "D" after video recording is restarted 



[0085] 

Thus, when video recording is paused during recording, since the music "D" is forcibly 
stopped at that point at which video recording was stopped, it sounds like the sound was cut off 
during video recording. However, even if video recording is performed between temporary stops 
during recording, since the Note ON and Note OFF are recorded linked to the audio track, 
unrelated to the temporarily paused interval, as shown in Figure 13(b), the sound of music "D" 
generated by the Note ON is not cut off midway during playback, but continues to play and stops 
normally with the subsequent Note OFF. 

[0086] 

In this case as well, as in the example embodiment, since control signals to cause the sound 
source to generate sound are written, without writing the sound signal itself (audio waveform), the 
"pop" noises at the seams between the point at which recording was paused and the point at which 
recording was restarted are not generated. 

[0087] 

In addition, whether video recording by the video camcorder was underway or stopped was 
detected in the embodiment described above by monitoring a synchronization signal in the video 
image signal, and operation of the sequencer was controlled according to this. In other words, this 
simultaneous recorder controls operation of the sequencer by tracking the operation of the video 
camcorder. However, this invention is not limited to this, and conversely, the operation of the 
sequencer and the operation of the video camcorder could also be centrally controlled by a 
controller (CPU) in the simultaneous recorder. For example, if instructions to start/stop video 
recording, etc., were given by remote control input to the video camcorder, operation of the video 
camcorder and sequencer could be synchronized even without monitoring the operation of the 
video camcorder in the simultaneous recorder by means of synchronization detection, etc. 

[0088] 

Further, since the video camcorder needs time for the mechanical operations from when it 
is instructed to record video until video recording is actually started in this condition, there may be 
situations in which operation of the sequencer precedes the video camcorder starting video 
recording, but in this case, the controller may be configured to perform processing to delay the 
instruction to the sequencer to start or stop playing by a specified time. 



[0089] 

There will probably also be cases in which, if video recording by the video camcorder is 
stopped while video recording is paused, the BGM performance data signal will not be recorded to 
the end. For instance, there is the case in which video recording by the video camcorder is stopped 
while the signal expressing "Note ON" in the BGM performance data is being recorded. In this 
case, the controller could monitor the operation of the sequencer and, if the sequencer is in the 
middle of generating performance data, such as "Note ON", etc., instruct the video camcorder to 
stop video recording after waiting for this to be completed. 

[0090] 

By controlling the start/stop timing of the operation of the sequencer and video camcorder, 
the controller can prevent the loss of the performance data being recorded. 

[0091] 

In each of the example embodiments discussed above, the simultaneous recorder of this 
invention is described as being constructed as a separate device from the video camcorder being 
used mounted onto the video camcorder, but this invention is of course not limited to this and may 
also be incorporated into the video camcorder as a single unit. In addition, since the user does not 
necessarily have to hear the BGM song during video recording, the control signal transmission 
mechanism that sends the control signal for simultaneous recording can be separated as a 
stand-alone device from the performance sound generating mechanism of the sequencer and sound 
source, so that simultaneous recording is accomplished by attaching only the former control signal 
transmission mechanism to the video camcorder during video recording, and playing the BGM 
song together with video playback by attaching the latter performance sound generating 
mechanism only during playback. This would enable the device attached to the video camcorder 
during video recording to be made smaller and easier to handle. 

[0092] 

Effect of the invention 

With this invention, as described above, BGM songs flowing in the background can be 
continuous, even at the transitions between scenes in the recorded video image, making it possible 
to prevent discontinuity in the song and noise from getting mixed in. 

Brief description of the figures 

Figure 1 is an explanatory diagram of this invention. 



Figure 2 is a diagram showing the block structure of a simultaneous music recording 
apparatus (simultaneous recorder) as an embodiment of this invention. 

Figure 3 is a drawing to explain the condition in which the simultaneous recorder of this 
embodiment is attached to the video camcorder. 

Figure 4 is a diagram showing an external view of the control group of the simultaneous 
recorder of this embodiment. 

Figure 5 is a diagram showing the circumstances of recording onto the recording tracks in 
order to explain the operation of the simultaneous recorder of this embodiment. 

Figure 6 is a flow chart of the routine executed when the power is turned on to the 
simultaneous recorder of this embodiment. 

Figure 7 is a flow chart of the timer interrupt routine of the simultaneous recorder of this 
embodiment. 

Figure 8 is a flow chart of the REC mode routine (1/3) of the simultaneous recorder of this 
embodiment. 

Figure 9 is a flow chart of the REC mode routine (2/3) of the simultaneous recorder of this 
embodiment. 

Figure 10 is a flow chart of the REC mode routine (3/3) of the simultaneous recorder of this 
embodiment. 

Figure 11 is a flow chart of the PLAY mode routine (1/2) of the simultaneous recorder of 
this embodiment. 

Figure 12 is a flow chart of the PLAY mode routine (2/2) of the simultaneous recorder of 
this embodiment. 

Figure 13 is an explanatory diagram of the circumstances of recording onto the recording 
tracks in order to explain the simultaneous recorder of another embodiment of this invention. 
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Figure 1 



Key: a Point at which shooting was stopped/restarted between scene A and scene B 

b Video signal (Video track) 

c Video recording tracks 

d Control signal (audio track, etc.) 

e Audio signal (audio track) 

f (a and b are continuous music) 

g Music performance status 

h Start music performance point 

i Performance control information (song selection, start performance indicator, etc.) 
or other subsequent performance data (Note ON/OFF), etc. 
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Figure 2. Block structure of embodied apparatus 



Key: a INPUT OUTPUT 
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b Inside [ ] . . . Structure of this apparatus 

1 CONTROLLER (CPU) 

2 CONTROL GROUP 

3 SEQUENCER 

4 SONG DATA ROM 

5 SOUND SOURCE 

6 MIXER 
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Figure 3. Mounting condition of embodied apparatus 
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Figure 4. External view of control group 




Key: [A] DURING VIDEO RECORDING 
a AUDIO TRACK CHI (L) 

AUDIO TRACK CH2 (R) 

VIDEO TRACK 
al PAUSE VIDEO RECORDING 
b RECORDED ACTIVE SIGNAL 

RESTART VIDEO RECORDING 
c SCENE A 
d SCENE B 
e VIDEO 

f RECORDED START SIGNAL 
g START PLAY 

h SONG SELECTOR BUTTON PRESSED 

i STOP PLAY 

j RESTART PLAY 

k STOP PLAY 

L SEGMENT IN WHICH ACTIVE SIGNAL WAS NOT RECORDED 
m STOP BUTTON PRESSED 



n TIME 

[B] DURING PLAYBACK 

o SCENE A 

p SCENE B 

q VIDEO 

r START PLAY 

s MUSIC PERFORMANCE IS TIED TOGETHER 

t SEGMENT IN WHICH BGM PERFORMANCE IS STOPPED BECAUSE, 
EVEN AFTER WAITING, NEXT ACTIVE SIGNAL IS NOT RECORDED 

u STOP PLAY 





Figure 6. Routine at power-up 



Key: A INITIALIZATION PROCESSING 

B RUN TIMER INTERRUPT ROUTINE 

C MODE SW: REC? 

D TO REC ROUTINE 

E TO PLAY ROUTINE 




Figure 7. Timer interrupt routine 
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Figure 8. REC mode routine (1/3) 
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Figure 9. REC mode routine (2/3) 

Key: D13 SONG SELECTOR BUTTON PRESSED? 

D14 GET SONG NUMBER CORRESPONDING TO SONG SELECTOR BUTTON 

D15 SEND START SIGNAL AND SONG NUMBER TO ENCODER 

D16 SEND SONG NUMBER TO SEQUENCER, INSTRUCT TO START 

D18 CONTINUE FLAG = 1 ? 

D19 INSTRUCT SEQUENCER TO CONTINUE-START 

D2 1 CONTINUE FLAG <- 0 

D22 PLAY FLAG <r- 1 




Figure 10. REC mode routine (3/3) 



Key: D23 PLAY FLAG = 1 ? 

NO (CASE 4) 

YES (CASE 3) 
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NO (TO PLAY ROUTINE) 
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Figure 11. PLAY mode routine (1/2) 



Key: El PLAY FLAG <- 0 

E2 SIGNAL ARRIVED FROM DECODER? 

E4 PLAY FLAG = 1 ? 

E5 INSTRUCT SEQUENCER TO STOP 

E6 PLAY FLAG <- 0 

E7 IS SIGNAL START SIGNAL? 

E8 PLAY FLAG = 1 ? 

E9 EXTRACT SONG NUMBER FROM DECODER SIGNAL 

E10 SEND SONG NUMBER TO SEQUENCER, INSTRUCT TO START 

Ell PLAY FLAG <- 1 

E12 IS SIGNAL ACTIVE SIGNAL? 




Figure 12. PLAY mode routine (2/2) 
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Figure 13 



Key: a AUDIO TRACK CHI 

b PAUSE VIDEO RECORDING 

bl RESTART VIDEO RECORDING 

c VIDEO TRACK 

d AUDIOT TRACK CH2 

e "C" NOTE ON 

f "C" NOTE OFF 

g "D" NOTE ON 

h "D" NOTE OFF 

i TIME 
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